Update: elide A5 terminal releases after orchestration - #2070

Summary

This change removes A5 TensorMap-and-RingBuffer scheduler work that is no longer needed after orchestration has sealed the task graph.

• Once orchestrator_done is observed at an existing deferred-release boundary, schedulers stop performing per-task release and discard the local deferred-release backlog.
• After every scheduler has left dispatch successfully, one scheduler closes the remaining live ring slots in bulk and publishes the final ring state.
• Error paths and exits before orchestration is sealed keep the original exact-release behavior.
• Terminal closure is recorded as a separate scheduler profiling phase.

The implementation was rebuilt from current main; it does not add explicit hot-function alignment or a per-loop orchestrator_done load.

Why this is safe

orchestrator_done seals the task graph: no new tasks can arrive after that point. Per-task release normally reclaims ring slots so future submissions can reuse them. Once the graph is sealed, that incremental reuse is no longer needed.

The release work is skipped only for a successful sealed execution. The terminal protocol waits until:

1. orchestration is complete;
2. all submitted tasks are complete;
3. every scheduler has left dispatch; and
4. no scheduler has reported an error.

The final scheduler then closes all still-live slots and publishes the terminal ring state. Therefore the lifecycle is still closed before runtime teardown, while failed or unsealed executions retain exact release semantics.

Implementation

• Reuse existing deferred-release drain boundaries to read the seal, avoiding a new atomic read in every scheduler hot-loop iteration.
• Keep terminal coordination in two 32-bit atomics placed in the existing 8-byte SchedulerContext tail gap.
• Move successful terminal closure to a cold, out-of-line path.
• Cover normal completion, async completion, error fallback, and terminal-close lifecycle state with C++ unit tests.
• Add TerminalClose to A5 scheduler swimlane conversion and host collection, classify it as Scheduler work in overhead analysis, and make the A5 DFX smoke validate the new terminal phase.

A5 scope

This optimization is intentionally limited to A5 because the platform scope follows the observed bottleneck. The motivating A5 timelines contain a visible tail after the Orchestrator has finished submitting tasks: Schedulers continue performing per-task release even though no new task can consume the reclaimed capacity, and that work extends the execution critical path. No tail release blocking was found on A2/A3. There is therefore no performance evidence that A2/A3 would benefit from the additional graph-seal observation, terminal barrier, leader election, and bulk-closure state, so A2/A3 keeps its existing incremental-release lifecycle. This is an evidence-based software decision rather than an A5 hardware limitation and can be revisited if future A2/A3 timelines expose the same blocking pattern.

Performance

The paired A5 run used card 1. Non-Qwen cases ran 100 precise iterations; Qwen3 ran 5 iterations and used the middle three Device-ranked rounds. Each Before/After sample also had an independent level-4 swimlane and a separately collected deps.json.

The performance run predates the final rebase by one upstream commit: its baseline was bf68bb6c (main at measurement time). PR source is now rebased on 0616e925 (#2073); correctness checks below were rerun after that rebase.

For the seven non-Qwen cases, the Effective geometric-mean speedup is 9.195%. Under the benchmark acceptance rule, the 5%-10% geometric-mean tier permits at most one case with an Effective regression of 5% or more; this run has zero. Qwen3 improves by 0.604%, satisfying its less-than-2% regression gate.

All eight coarse swimlane gates also pass. Full AICore-window deltas range from -29.260% to +0.945%. benchmark_bgemm has a diagnostic Orchestrator increase of 8.514%, but its acceptance metric, Effective, regresses by 4.692% and remains below the 5% case threshold.

Swimlane evidence

paged_attention_unroll Case1 provides a clear visual example of the optimization. In the Main timeline, the Orchestrator has already stopped submitting work while all four Scheduler threads continue through long terminal release regions. In the PR timeline, that terminal per-task release tail is absent; useful scheduling continues until execution converges, and lifecycle closure is handled by the terminal protocol.

Main (Before)

PR #2070 (After)

The yellow regions are Scheduler release work. Before the change, these regions form the visible post-orchestration tail; after the change, no corresponding terminal release tail remains.

Validation

• pre-commit run --from-ref upstream/main --to-ref HEAD: passed
• no-hardware C++ unit tests: 126/126 passed
• profiling analysis and swimlane converter Python unit tests: 49/49 passed
• A5 simulator chip-swimlane smoke: 4/4 passed
• A5 simulator DFX-off runtime build: passed
• A5 coarse benchmark: 8 Before + 8 After runs passed
• profiling artifacts: 16 swimlanes, 16 independent dependency captures, and 16 merged traces generated
• A5 precise benchmark: all non-Qwen cases 100 iterations; Qwen3 5 iterations

Local reproduction (A5 8P)

Independent re-measure on a local A5 8P host (Ascend950PR, device 4). Baseline is merge-base 0616e925; PR is 2d233a2fe. Non-Qwen cases ran 100 iterations (Avg Effective); Qwen3 ran 5 iterations and used the median Effective.

For the seven non-Qwen cases, the Effective geometric-mean speedup is 8.296%. Zero cases have an Effective regression of 5% or more. Qwen3 improves by 2.253%, satisfying its less-than-2% regression gate.